1. Field of Invention
The invention relates to a method of processing images in order automatically to detect key points situated on the contour of an object, referred to as key pixels in an image referred to as the initial image.
The invention also relates to a device for implementing this method.
The invention finds its application in the automatic analysis of a cardiological image of the left ventricle.
2. Description of the Related Art
A cardiological image can favorably be obtained by a system for forming digitized X-ray images.
A cardiological image can also be obtained by means other than X-rays, and the method of processing images according to the invention shall be applicable thereto, since it does not depend on the process by which the image was obtained.
The process used most frequently hitherto to produce a cardiological image consists in injecting a contrast medium towards the left ventricle of a patient, and in recording a sequence of images of this left ventricle, over the duration of effectiveness of the contrast medium. These images, in particular, enable the practitioner to determine the volume of the left ventricle during a ventricular contraction or systole, and the volume of the left ventricle during a ventricular expansion or diastole. A quantity called the ejection fraction is then calculated as the difference between these volumes expressed as a percentage of the maximum volume of diastole. Knowledge of the ejection fraction is an important element in diagnosis by the practitioner. These images also enable the practitioner to determine two key points of the left ventricle, namely the points situated on either side of the aortic valve, which are fixed reference points, and to determine another key point which is the vertex of the left ventricle or apex, the position of which varies between the duration of diastole and the duration of systole. The evaluation of the variation in the distance between the apex and the aortic fixed points, which is called the shortening fraction, is also an important element in diagnosis by the practitioner.
Finally, the determination of the contour of the left ventricle enables the practitioner to detect possible anomalies of shape, and is also an important element in his diagnosis.
An automated approach to the detection of the left ventricle and in particular its contour, within a given image, is known from the prior art through the publication entitled "Left Ventricular Contour Detection": A Fully Automated Approach by PNJ van der ZWET et al. in PROCEEDING COMPUTERS IN CARDIOLOGY, pp. 359-362, Oct. 11-14, 1992, Durham, N.C. U.S.A., in IEEE Computer Society Press, Los Alamos, Calif., U.S.A. This known system uses a combination of algorithms to detect a simplified model of the contour of the left ventricle. The first algorithm, a pyramidal segmentation algorithm, segments the starting image into a fairly large number of regions formed of square boxes of homogeneous intensity. The second algorithm uses a neural network to select the regions which have the greatest likelihood of belonging to the left ventricle. These regions are next merged and then the pixels grouped together as all belonging to the left ventricle are extracted to form the model. After detection of the model, the actual contour is found by dynamic programming. The position of the aortic valve is deduced from the shape of the contour detected.
According to this known method, a pyramid of images consists of a number of images each half the size of the previous image. An image is generated by filtering, with a low-pass filter, the previous image and by sampling at a halved sampling frequency. In this way a stack of image layers can be obtained.
Firstly, the layers of the pyramid are generated by using low-pass Gaussian filters to prevent illegal sampling (or aliasing), and then the resampling of each layer with halved resolution is performed. The image of lowest resolution in the pyramid is a matrix of 16.times.16 pixels. Each image is segmented by using the segmentation of the previous images of lesser resolution as a guide. The image which has the lowest resolution is segmented by a technique of edge detection. According to the gradient in the low-resolution image, certain pixels are assigned the tag "edge". Groups of pixels which are completely surrounded by the "edge" pixels and which are linked with one another are tagged "isolated region". Finally, all the remaining "edge" pixels are tagged in at least one of the previously found regions, or else, if their grey level differs too much from any other surrounding region, they are allocated a new tag. The result of this segmentation contains a large number of regions consisting of just one pixel, especially in the vicinity of the edges. It is only within the most homogeneous regions of the image, inside the ventricle, the region of the lung or of the diaphragm, that regions of more than one pixel may be found.
Segmentation of the image with the lowest level of resolution is used to obtain segmentation of the images having a higher resolution. Each pixel in the image of highest resolution corresponds to a small number of pixels in the objects of the bottom-most layer. The pixel may only be assigned to one of these objects. A specific pixel is assigned to the most likely object in a statistical sense. It is possible, during the assigning of a tag, for the object to be apportioned to several groups of pixels without correlation in the image of highest resolution. A procedure of retagging subsequent to the assigning retags the apportioned object into a number of tags of different objects.
This procedure continues until the image of highest resolution has been segmented. The segmentation of the image of highest resolution is used in combination with a neural network to obtain the final segmentation.
This neural network is designed for detection of the position of the left ventricle in an image which has been reduced by low-pass filtering and 16.times.16 sub-sampling. This neural network has three layers. The first layer is the low resolution image which is obtained by low-pass faltering then resampling of the initial ventriculogram. The low-pass filtering is used to prevent illegal sampling (aliasing). The second and third layers each contain 256 neurons. Each of these neurons receives the input of 25 elements of the previous layer through a separate weight for each input. The sigmoid function used to finally calculate the output levels is a continuous function. This enables the neural network to be used as a control module within the model detection algorithm. The pyramidal segmentation algorithm used first results in many different regions, all of which ought to be included within the final left ventricle. Moreover, the resolution of the results from the neural network is too low (16.times.16) to define the ventricular model properly. Furthermore, the results of the segmentation are sensitive to the value of the threshold, this being required in order to isolate the ventricle from the background.
By combining the results of the two algorithms, better segmentation has been obtained according to this cited document.
For each object tagged in the pyramidal segmentation, a mean value is calculated. The result from the neural network is extended to the resolution of the highest layer in the pyramid. The mean of each object is then calculated by summing the values of the results from the neural network and dividing by the number of pixels in the object. Extensive regions in the ventricle will correspond to points of the neural network with high probabilities. The same is true for the large regions outside the ventricle. It is only on the edges of the ventricle that the pyramidal segmentation consists of a large number of small regions. Their probability of being incorporated will depend on the value of the single neuron with which they correspond. Even if a few errors may be made, assigning to the ventricle an object which ought not to be so assigned, this has no great influence on the model finally detected.
The model is finally obtained by combining all the regions internal to the ventricle, and by extracting the remainder of the image therefrom. At the end of the processing there therefore remain just two classes: the pixels belonging to the ventricle and the pixels not belonging thereto.
Having once found the model, dynamic programming techniques are applied to detect the contour of the left ventricle; these techniques are known to those skilled in the art.
The algorithm taught by the cited publication uses the curves of the contour to determine the position of the aortic valve. To do this it is assumed that the ends of the valve are contour points having a maximum curvature of approximately 90.degree.. In order to avoid choosing other maxima as valve endpoint, the calculation of the angle of the valve endpoints is weighted. For ventriculograms this angle is approximately the same for all patients.
Finally, the valve endpoints and the contour of the ventricle are displayed and the user can correct the contour and retrace the valve. Following which, the dynamic programming techniques are applied just to the corrected parts of the ventricle. When both the contour and the valve endpoints have been determined for the duration of diastole and the duration of systole, and after the user has made corrections, the technical parameters are then calculated.
A drawback of the system described in the cited document is that it uses a very sizeable neural network having a large number of inputs, 256 inputs, and a large number of outputs, 256 outputs, with very rich hidden layers. Hence the weights are difficult to determine and learning requires a very sizeable database.
Moreover, this known system ultimately provides the model only inaccurately, by classifying the pixels into just 2 classes, that of the internal pixels and that of the external pixels, and deduces only the location of 2 key pixels, namely the valve end pixels. The location of the apex is not provided automatically.
Given the ever greater demands of medical techniques, it has been found that the system described in the cited document is not accurate enough nor complete enough and that another solution must be sought.
Nowadays, in a novel approach, those skilled in the an are envisaging a process for achieving analysis of images of the left ventricle involving two main steps:
a first step of locating the three key points formed by the two ends of the aortic valve and by the apex, which step can currently be effected in a sufficiently accurate manner only if it is conducted by hand on a ventriculogram by the practitioner himself, and therefore necessarily entails human interaction, PA1 a second step in which the accurate data of the three key points are provided as starting dam, for example manually by an operator, by means of a mouse to a computer, at the same time as the stored data relating to the intensity values linked with each pixel of the original image. The positions of the pixels located on the contour of the left ventricle are then calculated by means of already-known algorithms. PA1 the two end points of the aortic valve, PA1 the vertex of the left ventricle or apex. PA1 storage in the initial image in the form of a two-dimensional matrix of pixels of the intensity values of each pixel labelled by its coordinates, PA1 storage in the initial image of data of regions of the object which are referred to as classes including classes which are referred to as corresponding classes respectively containing the key pixels to be detected; PA1 selection of pixels of the initial image which are referred to as pixels of interest, on the contour, inside and outside the object; PA1 generation of a first vector of characteristics for each of the pixels of interest; PA1 classification of the pixels of interest into said classes of the object on the basis of their respective vector of characteristics. PA1 selection of each of the key pixels from among the pixels of interest of the corresponding class. PA1 first storage means for storing, in the initial image in the form of a two-dimensional matrix of pixels, the intensity values of each pixel labelled by its coordinates, and data of regions of the object, which are referred to as classes, including classes referred to as corresponding classes respectively containing the key pixels to be detected; PA1 means of selection of pixels of the initial image, referred to as pixels of interest, on the contour, inside and outside the object; PA1 first means of calculation, for generating a first vector of characteristics for each of the pixels of interest; PA1 a first neural network for receiving at its inputs the first vector of characteristics and for performing a classification of the pixels of interest into said classes of the object on the basis of their respective vector of characteristics.
When the data relating to the location of the three key points are provided accurately in the first step, the second step then poses no problem and the result is that the contour of the left ventricle is obtained with all the desired accuracy.